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- The MAILING DATE of this communication appears on the cover sheet with the correspondence address - 
Period for Reply 



A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 .136(a). tn no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 
Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1.704(b). 

Status 

I) K Responsive to communication(s) filed on 23 December 2004 . 
2a)M This action is FINAL. 2b)Q This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 11, 453 O.G. 213. 

Disposition of Claims 

4) £3 Claim(s) 1-4 and 6-15 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) D Claim(s) is/are allowed. 

6) E3 Claim(s) 1-4. 7, and 11-15 is/are rejected. 

7) 03 Claim(s) 8-10 is/are objected to. 

8) D Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) D The specification is objected to by the Examiner. 

10)[3 The drawing(s) filed on 25 May 2000 is/are: a)!3 accepted or b)D objected to by the Examiner. 
Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1.85(a). 
Replacement drawing sheet(s) including the correction is required if the.drawing(s) is objected to. See 37 CFR 1.121(d). 

II) D The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-152. 

Priority under 35 U.S.C. § 119 

12)D Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 119(a)-(d) or (f). 
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1 .□ Certified copies of the priority documents have been received. 

2. Q Certified copies of the priority documents have been received in Application No. . 
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application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 
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DETAILED ACTION 

The examiner notes that this office action is a follow up on the interview summary 
dated January 27, 2005, in which the previous office action was vacated. 

Claim Objections 
. Claims 9 and 15 are objected to because of the following informalities: 

Claim 9 is written to be dependent on claim 1 , and recites the limitation "pixels 
within each designated structure," in line 2. This limitation lacks antecedent basis as 
written to depend on claim 1. The examiner believes the claim should depend on claim 
8, and will be examined as understood by the examiner. 

Claim 15 recites the limitation of a "extracting a first individual frame and a 
second individual frame of from a series of video frames," in lines 6-7. The examiner 
believes this should read, "extracting a first individual frame and a second individual 

frame from a series of video frames," and will be examined as understood by the 

✓ 

examiner. 

Appropriate correction is required. 

Claim Rejections - 35 USC § 103 

The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 
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1. Claims 1-4, 7, and 11-15 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Hsieh et al. (US Patent # 6,01 1 ,558) in view of Burt et al. (US Patent 
# 5,999,662). 

2. In regard to claim 1 , note Hsieh discloses the use of extracting a first individual 
frame and a second individual frame of imagery (column 4, lines 33-35), detecting 
edges in the first individual frame and the second individual frame (column 4, lines 9-19; 
the edges in the two images are detected), determining regions of interest in the first 
individual frame and the second individual frame based on the detected edges (column 
4, lines 9-19; the matched positions are considered to be the regions of interest), 
identifying commonality from the first individual frame to the second individual frame, 
including correlating determined regions of interest between the two individual frames 
by comparing each region of interest in the first individual frame to a region of interest in 
the second individual frame (column 4, lines 9-19; the matched positions are considered 
to be the regions of interest, which are compared using the hypothesis generator in 
order to match them properly), overlapping the individual frames based on the 
commonality identified from the first individual frame to the second individual frame 
(column 4, lines 9-19; the images are aligned into one and blended), and displaying an 
image representing a continuous area (column 6, lines 19-21). Therefore, it can be 
seen that the Hseih device fails to disclose that the images used are extracted from a 
series of video frames. Burt discloses the use of an image mosaic system that uses 
images extracted from a series of video frames (column 17, lines 36-54). It is well 
known in the art that the use of sequential images from a video sequence can be 
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stitched together in order to form a panoramic image or virtual reality environment. 
Therefore, it would have been obvious to one of ordinary skill in the art to modify the 
Hsieh device to include the use of images extracted from a series of video frames in 
order to stitch together related images to form a panoramic image or virtual reality 
environment. 

1 . In regard to claim 2, note the primary reference of Hsieh in view of Burt discloses 
the use of a method comprising extracting individual frames of imagery taken from 
video, identifying commonality from one frame to the next, and overlapping the 
individual frames and displaying and. image representing a continuous area. Therefore, 
it can be seen that the primary device lacks the use of a camera that takes images at 30 
frames per second. Official notice js taken that the concepts and advantages of using a 
camera that takes images at 30 frames per second are notoriously well known and 
expected in the art. Therefore, it would have been obvious to one of ordinary skill in the 
art to modify the primary device to include the use of a video camera that takes images 
at 30 frames per second in order to allow the video to also be displayed on a 
conventional television. 

2. In regard to claim 3, note Burt discloses the use of MPEG compression to store 
the images (column 15, lines 3-6). 

3. In regard to claim 4, note Burt discloses the conversion of MPEG files into black 
and white images (column 5, lines 7-12). 

4. In regard to claim 7, note Burt discloses the compensation of platform/camera 
motions (column 19, lines 12-15). 
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5. In regard to claim 1 1 , note Hsieh discloses the correlation of regions of interest 
by comparing each region of interest to each other region of interest (column 4, line 9 
column 5, line 5; in order for the hypothesis generator to generate an optimal 
hypothesis, each region of interest has to be compared to determine the best possible 
match) 

6. In regard to claim 12, note Hsieh discloses calculating a centroid for each region 
of interest in the first individual frame (column 4, lines 60-66; the elements in the first 
image, P a , are used to calculate the centroid region of interest, position k), comparing 
the centroid in the first individual frame with all centroids of the second individual frame 
(the equation in column 4, lines 55-60 uses k to compare the two image centroids), 
selecting centroids in the second individual frame within an error tolerance (column 3, 
lines 8-1 1 ; optimum correction requires absolute error correlation techniques), 
correlating an average distance from every pixel in a region of interest in the first frame 
with every pixel in a corresponding region of interest in the second individual frame 
(column 4, lines 20-67), determining the most consistent average distance between a 
region of interest in the first frame and a corresponding region of interest in the second 
frame (column 4, line 20-column 6, line 10; the most consistent average distance 
between images is calculated using equation l(n)), wherein the overlapping step is 
performed based on the determined most consistent average distance (column 5, line 
63- column 6, line 10; based on the distance between the most common parts of the 
image, the image is overlapped using equation l(n)). 



Application/Control Number: 09/577,487 Page 6 

Art Unit: 2612 

7. In regard to claim 13, this is an apparatus claim, corresponding to the method in 
claim 1. Therefore, claim 13 has been analyzed and rejected as previously discussed 
with respect claims 1 . 

8. In regard to claim 14, this is an apparatus claim, corresponding to the method in 
claim 1. Therefore, claim 14 has been analyzed and rejected as previously discussed 
with respect claims 1. 

9. In regard to claim 15, this is an apparatus claim, corresponding to the method in 
claim 1. Therefore, claim 15 has been analyzed and rejected as previously discussed 
with respect claims 1 . 

Allowable Subject Matter 

10. Claims 8-10 are objected to as being dependent upon a rejected base claim, but 
would be allowable if rewritten in independent form including all of the limitations of the 
base claim and any intervening claims. 

11. As for claims 8, the prior art does not teach or fairly suggest the use of a video 
mosaic system that searches the image for adjacent "on" pixels until and "off" pixel is 
detected, counting the number of "on" pixels and designating these pixels as a structure 
if above a threshold. 

Conclusion 

The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure. 

US006597801B1: note the use of an object registration system. 
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Applicant's amendment necessitated the new ground(s) of rejection presented in 
this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP 
§ 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 
CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1 .136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the date of this final action. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Chriss S. Yoder, III whose telephone number is (703) 
305-0344 or (571) 272-7323. The examiner can normally be reached on M-F: 8 - 4:30. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Wendy Garber can be reached on (703) 305-4929. The fax phone number 
for the organization where this application or proceeding is assigned is 703-872-9306. 
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Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). 

CSY 

February 9, 2005 




